Successive Approximation Coding for Distributed Matrix Multiplication
نویسندگان
چکیده
Coded distributed computing was recently introduced to mitigate the effect of stragglers on systems. This paper combines ideas approximate and coded further accelerate computation. We propose successive approximation coding (SAC) techniques that realize a tradeoff between accuracy speed, allowing system produce approximations increase in over time. If sufficient number compute nodes finish their tasks, SAC exactly recovers desired theoretically provide design guidelines for our techniques, numerically show achieves better accuracy-speed comparison with previous methods.
منابع مشابه
Analysis of a randomized approximation scheme for matrix multiplication
This note gives a simple analysis of a randomized approximation scheme for matrix multiplication proposed by [Sar06] based on a random rotation followed by uniform column sampling. The result follows from a matrix version of Bernstein’s inequality and a tail inequality for quadratic forms in subgaussian random vectors.
متن کاملDistributed Adaptive Sampling for Kernel Matrix Approximation
Most kernel-based methods, such as kernel or Gaussian process regression, kernel PCA, ICA, or k-means clustering, do not scale to large datasets, because constructing and storing the kernel matrix Kn requires at least O(n2) time and space for n samples. Recent works [1, 9] show that sampling points with replacement according to their ridge leverage scores (RLS) generates small dictionaries of r...
متن کاملStraggler Mitigation in Distributed Matrix Multiplication: Fundamental Limits and Optimal Coding
We consider the problem of massive matrix multiplication, which underlies many data analytic applications, in a large-scale distributed system comprising a group of worker nodes. We target the stragglers’ delay performance bottleneck, which is due to the unpredictable latency in waiting for slowest nodes (or stragglers) to finish their tasks. We propose a novel coding strategy, named entangled ...
متن کاملCommunication lower bounds for distributed-memory matrix multiplication
We present lower bounds on the amount of communication that matrixmultiplication algorithms must perform on a distributed-memory parallel computer. We denote the number of processors by P and the dimension of square matrices by n. We show that the most widely-used class of algorithms, the so-called 2-dimensional (2D) algorithms, are optimal, in the sense that in any algorithm that uses only O(n...
متن کاملSpectrum Approximation Beyond Fast Matrix Multiplication: Algorithms and Hardness
Understanding the singular value spectrum of a matrix A ∈ Rn×n is a fundamental task in countless numerical computation and data analysis applications. In matrix multiplication time, it is possible to perform a full SVD of A and directly compute the singular values σ1, . . . , σn. However, little is known about algorithms that break this runtime barrier. Using tools from stochastic trace estima...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: IEEE journal on selected areas in information theory
سال: 2022
ISSN: ['2641-8770']
DOI: https://doi.org/10.1109/jsait.2022.3190859